iCAM06, HDR, and Image Appearance

A new image appearance model, designated as iCAM06, has been developed for the applications of high-dynamic-range (HDR) image rendering and color image appearance prediction. The iCAM06 model, based on the iCAM framework, incorporates the spatial processing models in the human visual system for contrast enhancement, photoreceptor light adaptation functions that enhance local details in highlights and shadows, and functions that predict a wide range of color appearance phenomena. This paper reviews the concepts of HDR imaging and image appearance modeling, presents the specific implementation framework of iCAM06 for HDR image rendering, and provides a number of examples of the use of iCAM06 in HDR rendering and color appearance phenomena prediction. Introduction In recent years, high-dynamic-range (HDR) imaging technology 1,2,3 has advanced rapidly such that the capture and storage of a broad dynamic range of luminance is now possible, but the output limitations of common desktop displays have not followed the same advances. Although HDR displays4 will be more widely available in the near future, currently they are still costly, and for some applications such as hard copy printing the need for dynamic range reduction will always exist. HDR rendering algorithms, which are also known as tone-mapping operators (TMOs), are designed to scale the large range of luminance information that exists in the real world so that it can be displayed on a device that is only capable of producing a much lower dynamic range. Many of these algorithms have been designed for the single purpose of rendering high dynamic range scenes onto low dynamic range displays. Image appearance models, such as iCAM, attempt to predict perceptual responses to spatially complex stimuli. As such, they can provide a unique framework for the prediction of the image appearance of high dynamic range images. The iCAM06 model, which is based on the iCAM framework, was developed for HDR image rendering.5 A number of improvements5 have been implemented in iCAM06 on the motivation of a better algorithm that is capable of providing more perceptually accurate HDR renderings and a more developed perceptual model for a wide range of image appearance prediction. High-Dynamic-Range Image Perception and Rendering In real-world scenarios we might encounter a large range of luminance, from below 0.0001 candelas per meter-square (cd/m2) for typical starlight illumination, to over 10,000 cd/m2 for direct sunlight.6 Although it is unlikely that the sun and stars are in the same scenario, most outdoor scenes have dynamic ranges of 4-5 orders of magnitude from highlights to shadows.7 We have no problem in perceiving this luminance range in a single view due to local light adaptation in our visual systems. However, most common desktop display devices can only reproduce a moderate absolute output level and a limited dynamic range of about 100:1. HDR image rendering is necessary to ensure that the wide range of light in a real-world scene is conveyed on a display with limited capability. In addition, accurate reproduction of the visual appearance of the scene is required, resulting in an image that invokes the same responses as someone would have when viewing the same real-world scene. In the human visual system, the output responses of the retina have a contrast ratio of around 100:1, and the signal-to-noise ratio of individual channels in the visual pathway (from retina to brain) is about 32:1, less than 2 orders of magnitude.8 There are two classes of retinal photoreceptors, rods and cones. Rods are responsible for vision at low luminance levels (e.g. less than 1 cd/m2), while cones are less sensitive and are responsible to vision at higher luminance levels. Cones and rods have the same response curves, covering a range of about 3 log units.8 The transition from rod to cone vision is one mechanism that allows the human visual system to function over a large range of luminance levels. When the photoreceptors are continuously exposed to high background intensities, their sensitivities gradually decrease and the initial saturated response does not continue to remain saturated. This process is known as photoreceptors’ light adaptation and modeled by the Michaelis-Menten equation (or Naka-Rushton equation).9 Previous human vision research has shown that, given sufficient time to adapt, the intensity-response curves have the same shapes and maintain the log-linear property for about 3 log units of intensity range at any background intensity. Photoreceptor adaptation plays an important role in HDR perception. Color Appearance and Image Appearance The appearance of a given color stimulus depends on the viewing conditions in which it is seen. As far as the global conditions under which a color is viewed are concerned, there are a few factors affecting its appearance. The level of luminance has significant effects on perceptual colorfulness and contrast. An increase in luminance level results in an increase in perceived colorfulness (Hunt effect) and lightness contrast (Stevens effect). The lightness of the surround also influences image contrast, which is smaller when it is dim or dark (known as Bartleson-Breneman equations), and colorfulness, which is larger in a dim or dark surround. Chromatic adaptation in human visual system enables the ability of discounting the illuminant, allowing observers to perceive colors of objects more independent of changes of the illuminants. As for the effect of a color’s local surroundings, simultaneous contrast phenomenon is most significant. Simultaneous contrast causes a color to shift in color appearance when the background is changed. The apparent color shifts follow the opponent theory of color vision in a contrasting sense; in other words, a light background induces a color to appear darker, a darker background induces a lighter appearance, red induces green, green induces red, yellow induces blue, and blue induces yellow. Josef Albers patterns are one of the examples to demonstrate the interaction of color. A related phenomenon is crispening. Crispening refers to the effect that the perceived magnitude of color differences increase when viewing against a background similar to them. When the stimuli increase in spatial frequency, the stimuli’s color apparently mixes with its surround, which is called spreading. An overview of these color appearance phenomena is given by Fairchild.12 Color appearance models were developed for the prediction of color appearance across changes in media and viewing conditions. The research of color appearance modeling has culminated with a recommendation of the CIECAM97s in 199710 and its revision, CIECAM02, in 2002.11 Details on the formulation and evolution of these models can be found in Fairchild.12 While color appearance models are very useful in color reproduction across different media, they are limited in scope and are not designed for prediction of visual appearance of complex spatially varying stimuli such as images or video. An image appearance model extends color appearance models to incorporate properties of spatial and temporal vision allowing prediction of appearance of complex stimuli. Given an input of images and viewing conditions, an image appearance model can provide perceptual attributes of each pixel and describe human perception of the image. The inverse model can take the output viewing conditions into account and thus generate the desired output perceptual effect. An image appearance model should not be only limited to the traditional color appearance correlates such as lightness, chroma and hue, but rather those image attributes such as contrast and sharpness. The iCAM model, developed by Fairchild and Johnson,13,14,15 has demonstrated its potential in a broad range of image applications, such as image difference and image quality measurement, color appearance phenomenon prediction and HDR image rendering. The framework of iCAM has provided a powerful and clear scope for the development of further comprehensive image appearance models. The iCAM06 model, a new image appearance model based on iCAM, incorporates the spatial processing models in the human visual system for contrast enhancement, photoreceptor light adaptation functions that enhance local details in highlights and shadows, and functions that predict a wide range of color appearance phenomena.5 The iCAM06 Framework The goal of the iCAM06 model is to accurately predict human visual attributes of complex images in a large range of luminance levels and thus reproduce the same visual perception across media. Figure 1 presents the general flowchart of iCAM06 as applied to HDR image rendering originally presented by Kuang et al.5 A description of the model with example images and source code can be found at www.cis.rit.edu/mcsl/icam06. Note, that although the iCAM06 framework described in this section focuses on HDR image rendering, the parameters or modules can be specifically tuned for a wide range of situations, including, but not limited to, image appearance prediction. Figure 1 Flowchart of iCAM06 The input data for iCAM06 model are CIE tristimulus values (XYZ) for the stimulus image or scene in absolute luminance units. The absolute luminance Y of the image data is necessary to predict various luminance-dependent phenomena, such as the Hunt effect and the Stevens effect. Besides, the low-passed Y image is used in the whole image rendering chain to control the prediction of chromatic adaptation, image contrast and local details. A typical linear RGB encoded HDR image can be transformed into CIE 1931 XYZ tristimulus values through the specific camera characterization or through sRGB transformation by default. Once the input image is in device independent coordinates, the image is decomposed into a base layer, containing only large-scale variations, and a detail layer. The modules of chromatic adaptation and tone-compression processing are only applied to the base layer, thus preserving details in the image. The two-scale decomposition is motivated by two widely accepted assumptions in human vision: 1) An image is regarded as a product of the reflectance and the illuminance, and human vision is mostly sensitive to the reflectance rather than the illumination conditions; 2) human vision responses mostly to local contrast instead of the global contrast. These two assumptions are actually closely related since the local contrast is typically related to the reflectance in an image in some way. The fact that the human visual system is insensitive to the global luminance contrast enables the solution of compressing the global dynamic range and preserving local details in an HDR scene to reproduce the same perceptual appearance on a low-dynamic-range display that has a significantly lower maximum absolute luminance output. The base layer is obtained using an edge-preserving filter called the bilateral filter, previously proposed by Durand and Dorsey.16 The bilateral filter is a non-linear filter, where each pixel is weighted by the product of a Gaussian filtering in the spatial domain and another Gaussian filtering in the intensity domain that decreases the weight of pixels with large intensity differences. Therefore, bilateral filter effectively blurs an image while keeps sharp edges intact, and thus avoid the “halo” artifacts that are common for local tone-mapping operators. The base layer image is first processed through chromatic adaptation. The chromatic adaptation transformation embedded in iCAM, which is originally from CIECAM02, has been adopted in the iCAM06 model. It is a linear von Kries normalization of the spectrally sharpened RGB image signals by the RGB adaptation white image signals derived from the Gaussian low-pass adaptation image at each pixel location (RwGwBw). The amount of blurring in the low-pass image is controlled by the half-width of the filter ! " , which is suggested to set to 5-degree radius of background.17 The characteristics of the viewing conditions are often unknown for HDR image rendering application; thus a simplifying assumption can be to specify the width of the filter according to the image size itself. The iCAM06 model is extended to luminance levels ranging from low scotopic to photopic bleaching levels. The post-adaptation nonlinear compression is a simulation of the photoreceptor responses, i.e. cones and rods. Therefore, the tone compression output in iCAM06 is a combination of cone response and rod response. The CIECAM02 post-adaptation model is adopted as cone response prediction in iCAM06 since it was well researched and established to have good prediction of all available visual data. The CIECAM02 model uses a nonlinear compression to convert the physical metrics into perceptual dimensions. Instead of using a global source white, the iCAM06 transform uses a low-passed version of the absolute Y image as local adapted white; furthermore, iCAM06 provides a user-controllable variable to tune the steepness of the response curves (Figure 2), which enables to change the overall image contrast of tone mapping of the highdynamic-range images. The rod response functions are adapted from those used in the Hunt Model18 by using the same nonlinear response function as cones. The final tone compression response is a sum of cone and rod responses, illustrated in Figure 2. Figure 2 Cone and Rod responses after adaptation plotted against log luminance (log cd/m) for three adaptation levels (1, 3, 5 for cone and -1, 0, 1 for rod) in iCAM06. Open circles: reference whites; filled circles: adapting